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What is an algorithm? 


An algorithm is a sequence of 
unambiguous instructions for solving a 


problem, i.e., tor obtaining a required 
output for any legitimate input in a 
finite amount of time. 


j 


algorithm 
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Algorithm 


-= An algorithm is a sequence of 
unambiguous instructions for solving 


a problem, i.e., for obtaining a 
required output for any legitimate 
input in a finite amount of time. 


e Can be represented various forms 
e Unambiguity/clearness 

e Effectiveness 

e Finiteness/termination 


- : : e Correctness 
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Historical Perspective 


-= Euclid’s algorithm for finding the greatest 
common divisor 


-= Muhammad ibn Musa al-Khwarizmi - 9th 
century mathematician 
WWW ini ien hall/kh 
z.html 
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problem 


proble 
algorithm 
(or 
instance) 
Li algorithmic solution 
ea (different from a conventional solutiot 
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rxamptle oi computational 
problem: sorting 


Statement of problem: 
_ Input: A sequence of n numbers <a;, a», ..., an> 


- Output: A reordering of the input sequence <a), a 
2, +, an> SO that a;= a; whenever i < j 


=- Instance: The sequence <5, 3, 2, 8, 3> 


-= Algorithms: 
= Selection sort 
_ Insertion sort 
, Merge sort 
=- (many others) 
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Selection Sort 


- Input: array a[1],...,a[n] 


` Output: array a sorted in non-decreasing 
order 


_ Algorithm: 


a[n] 


~ ° Is this unambiguous? Effective? 
Duna e See also pseudocode, section 3.1 
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Problems 


= Sorting 

-= Searching 

' Shortest paths in a graph 

= Minimum spanning tree 

= Primality testing 

= Traveling salesman problem 
-= Knapsack problem 

Chess 

-= Towers of Hanoi 

-= Program termination 


Some of these problems don’t have efficient 
“= algorithms, or algorithms at all! 
Thay 
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Algorithms 


How to design algorithms 
` How to express algorithms 
' Proving correctness 


' Efficiency (or complexity) analysis 
- Theoretical analysis 


=- Empirical analysis 


= Optimality 
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Algorithm design strategies _ 


| 


- Brute force ' Greedy approach 
_ Divide and conquer | Dynamic programming 


Decrease and conquer Backtracking and branch- 


and-bound 
- Transform and conquer 


' Space and time tradeoffs 
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Analysis of Algorithms 


-= How good is the algorithm? 
= Correctness 
- Time efficiency 
= Space efficiency 


= Does there exist a better algorithm? 


=- Lower bounds 
= Optimality 
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What is an algorithm? 


Recipe, process, method, technique, 
procedure, routine,... with the following 
requirements: 
_ Finiteness 
=- terminates after a finite number of steps 
_ Definiteness 
= rigorously and unambiguously specified 
=- Clearly specified input 
valid inputs are clearly specified 
= Clearly specified/expected output 


can be proved to produce the correct output given a 
valid input 


- Effectiveness 
_ steps are sufficiently simple and basic 
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Why study algorithms? 


= Theoretical importance 


_ the core of computer science 


= Practical importance 
_ A practitioner’s toolkit of known algorithms 


-= Framework for designing and analyzing algorithms 
for new problems 


| Example: Google’s PageRank 
Ta Technology 
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Euclid’s Algorithm 


Problem: Find ged(im,n), the greatest common divisor 
of two nonnegative, not both zero integers m and n 


Examples: gcd(60,24) = 12, gcd(60,0) = 60, 
gcd(0,0) = ? 


Euchid’s algorithm is based on repeated application of 
equality 
gcd(m,n) = gcd(n, m mod n) 
until the second number becomes 0, which makes the 
problem 
trivial. 


a Example: gcd(60,24) = gcd(24,12) = gcd(12,0) = 12 
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LWO Gescriptions OL LUCIIQ S5 
algorithm 


Step 1 If n = 0, return m and stop; otherwise go to 
Step 2 
Step 2 Divide m by n and assign the value of the 
remainder to r 
Step 3 Assign the value of n to m and the value of r 
to n. Goto 
Step 1. 


while n 4 0 do 
r- m mod n 
m- n 
ner 

Sa return m 
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A dS | 
gcd(m,n) re 
Consecutive integer checking algorithm | 
Step 1 Assign the value of min{ m,n} to t 


Step 2 Divide m by t. If the remainder is 0, go 
to Step 3; 
otherwise, go to Step 4 
Step 3 Divide n by t. If the remainder is 0, 
return f and stop; 
otherwise, go to Step 4 


Hts RACE RL PEt 9° to Step 2 
algorithm? How much slower? 


= O(n), ifn <= m , vs O(log n) 
= 
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[cont.] 


Middle-school procedure 

Step 1 Find the prime factorization of m 
Step 2 Find the prime factorization of n 
Step 3 Find all the common prime factors 


Step 4 Compute the product of all the 
common prime factors 
and return it as gcd (m,n) 


Is this an algorithm? 


=p How eflicient ts ttfime complexity: 
~ O(sqrt(n)) 
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Sieve of Eratosthenes 


Input: Integer m= 2 
Output: List of primes less than or equal to n 
for p - 2 to n do Alp|<p 


for p - 2 to n do 


if Alpol 7 0 //p hasnt been previously eliminated from 
the list 


J= p*p 
while j= n do 
AJ — 0) //mark element as eliminated 
if Pp 
» Example:2 3 4 5 6 7 8 910 11 12 13 14 15 
om lË 17 18 1920 complexity: 
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algorithms 


… How to design algorithms 


… How to analyze algorithm efficiency 
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techniques/strategies 


_ Brute force ~ Greedy approach 


-= Divide and conquer -= Dynamic programming 


= Decrease and conquer } Iterative improvement 


= Transform and 


conquer -= Backtracking 
_ Space and time -= Branch and bound 
tradeoffs 
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Analysis of algorithms 


-= How good is the algorithm? 
+ time efficiency 
+ space efficiency 
- correctness ignored in this course 


| Does there exist a better algorithm? 
= lower bounds 
- optimality 
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Important problem types 


= sorting 


! searching 

_ string processing 

= graph problems 

= combinatorial problems 


-= geometric problems 


=- numerical problems 
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Sorting (I) 


Rearrange the items of a given list in ascending 
order. 
- Input: A sequence of n numbers <a;, az, ..., an> 
Output: A reordering <a, a2, ..., a ,> of the input 
sequence such that a;= as =. =a, 
: Why sorting? 
- Help searching 
=- Algorithms often use sorting as a key subroutine. 
' Sorting key 
- A specially chosen piece of information used to guide 
sorting. E.g., sort student records by names. 
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Sorting (II) 


' Examples of sorting algorithms 
Selection sort 
= Bubble sort 
_ Insertion sort 
= Merge sort 
-= Heap sort... 
= Evaluate sorting algorithm complexity: the number of 
key comparisons. 
= Two properties 
= 1: A sorting algorithm is called stable if it preserves 
the relative order of any two equal elements in its input. 
=: A sorting algorithm is in place if it does not 
see extra memory, except, possibly for a few memory 
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Selection Sort 


Algorithm SelectionSort(A[0..n-1) 

//The algorithm sorts a given array by selection sort 
//input; An array A[0..n-1] of orderable elements 
Output Array A[0..n-1] sorted in ascending order 
fori € 0 to n - 2 do 


min € i 
forj €&i+ 1ton -1 do 
if A[j] < A[min] 
min €j 
swap Ali] and A[min] 
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Searching 


’ Find a given value, calledia "#9 <-; ina 
given set. 
-= Examples of searching algorithms 
-= Sequential search 


i Input sored arty ai<...<ajand 
ey X; 


m €(i+j)/2; 
while i < j and x != a m do 
ifx <a mthenj € m-1 
else i € m+l; 
if x = a m then output a m; 
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String Processing 


= A string is a sequence of characters from an 
alphabet. 

-= Text strings: letters, numbers, and special 
characters. 

I F matching: searching for a given 

ai in a text. 


E 


(i) searching for a word or phrase on WWW 
or in a Word document 


(ii) searching for a short read in the 
reference genomic sequence 
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Graph Problems 


~ Informal definition 


+ A graph is a collection of points called 
vertices, some of which are connected by line 
segments called edges. 


` Modeling real-life problems 
-= Modeling WWW 
-= Communication networks 
-= Project scheduling ... 

-= Examples of graph algorithms 
= Graph traversal algorithms 
= Shortest-path algorithms 
_ Topological sorting 

= 
“Be 
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Fundamental data structures 


E 


list - graph 
array | tree and binary 
- linked list tree 
string ` set and dictionary 
= stack 
_ queue 
priority 
ua queue/heap 
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Linear Data Structures 


A Aimays: Arrays 
_A re of nes of the 7 " fixed length (need 
same data type that are store a 
contiguously in computer preliminary reservation of 
memory and made accessible by memory) 
die a value of the arrays "= contiguous memory locations 
ndex. l 
"Linked List direct access 
- A sequence of zero or more F " Insert/delete 
nodes each containing two kinds 
of information: some data and Linked Lists 
one or more links called = dynamic lenath 
pointers to other nodes of the y i 2 i 
linked list. = arbitrary memory locations 
_ Singly linked list (next pointer) = access by following links 
previous pointers) 


al Hal}. 
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Stacks and Queues 


' Stacks 
= A stack of plates 
Insertion/deletion.can be done only at the top. 
=- LIFO 
- Two operations (push and pop) 
_ Queues 
- À queue of customers waiting for services 


_ Insertion/enqueue from the rear and 
deletion/dequeue from the front. 
FIFO 
E -= Two operations (enqueue and dequeue) 
= | 
a 
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Priority Queue and Heap 


| 
heaps) 


= À data structure for maintaining a of 
elements, each associated with a key/priority, 
with the following operations 


Finding the element with the highest priority 
Deleting the element with the highest priority 


Inserting a new element 


=" Scheduling jobs on a shared compute 
~ 
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Graphs 


m Formal definition 
A graph G = <V, E> is defined by a pair of 
two sets: a finite set V of items called 
vertices and a set E of vertex pairs called 
edges. 
=- Undirected and directed graphs (digraphs). 
= What’s the maximum number of edges in an 
undirected graph with |V| vertices? 
' Complete, dense, and sparse graphs 
-= A graph with every pair of its vertices 
connected by an edge is called complete, 


Kvi 
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Graph Representation 


-~ Adjacency matrix 
= n xn boolean matrix if |V| is n. 


- The element on the ith row and jth column is 1 if 
there’s an edge from ith vertex to the jth vertex; 


otherwise 0. 
=- The adjacency matrix of an undirected graph is 
symmetric. 
! Adjacency linked lists 


_ A collection of linked lists, one for each vertex, that 
contain all the vertices adjacent to the list’s vertex. 


= Which data structure would you use if the graph is a 


100-node stay shape? whee E 
Hi Eg 
T~ 0000 ae 
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Weighted Graphs 


DS Wergittedigrapiis 
=- Graphs or digraphs with numbers assigned 
to the edges. 


5 
Sa 
8 
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Graph Properties -- Paths and 
Connectivity 


=- Paths 


- A path from vertex u to v of a graph G is defined as 
a Sequence of adjacent (connected by an edge) 
vertices that starts with u and ends with v. 


- Simple paths: All edges of a path are distinct. 


- Path lengths: the number of edges, or the number 
of vertices - 1. 


' Connected graphs 


- A graph is said to be connected if for every pair of 
its vertices u and v there is a path from u to v. 


' Connected er 


- Them cted graph of a given 
| graph. 
Pan 
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Graph Properties -- Acyclicity 


_ Cycle 
d A simple path of a positive length that 
starts and ends a the same vertex. 


= Acyclic graph 
_A graph without cycles 
+ DAG (Directed Acyclic Graph) 
ta 
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Trees 


' Trees 
- A tree (or free tree) is a connected acyclic graph. 


-= Forest: a graph that has no cycles but is not 
necessarily connected. 


_ Properties of trees 


< For every two vertices in a tree there always exists 
exactly one simple path trom one of these vertices 
to the other. Why? 
= Rooted trees The above property makes it 


possible to select an arbitrary ve rieng Rite 
tree and consider it as the root of GŒalled 
nn 


Hans 
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Rooted Trees (I) 


-~ Ancestors 
-= For any vertex vin a tree T, all the vertices on the 
simple path from the root to that vertex are called 
ancestors. 
Descendants 
= All the vertices for which a vertex vis an ancestor 
are said to be descendants of v. 
-= Parent, child and siblings 
If (u, v) is the last edge of the simple path from the 
root to vertex v, u is said to be the parent of vand v 
is called a child of u. 


= Vertices that have the same parent are called 
siblings. 


= Leaves 
A vertex without children is called a leaf. 
= Subtree 
nn. < A vertex v with all its descendants is called the 
= = subtree of T rooted at v. 
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Rooted Trees (II) 


= Depth of a vertex 


_ The length of the simple path from the root 
to the vertex. 


' Height of a tree 
_ The length of the longest simple path from 


the root to a leaf. 
hi = 
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Ordered Trees 


- Ordered trees 


= An ordered tree is a rooted tree in which all the children of 
each vertex are ordered. 


Binary trees 

_ A binary tree is an ordered tree in which every vertex has 
no more than two children and each children is designated 
s either a left child or a right child of its parent. 

biliary, search trees 

- Each vertex is assigned a number. 

-= A number assigned to each parental vertex is larger than 
all the numbers in its left subtree and smaller than all the 
numbers in its right subtree. 

© logn = 4 = no- 1, where h is the height of a binary tree and n 
the size. 
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